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ABSTRACT 



The effects of a conflict resolution program on school 
climate, student self-image, and the use of conflict resolution skills were 
studied in urban schools in Canada. Benefits and limitations of conflict 
resolution in comparison with other types of violence prevention programs and 
methods of evaluating violence prevention programs were also studied. The 
context was grade-5 classrooms of 3 French and 4 English elementary schools 
in 2 urban areas of a large Canadian city, for a total of 140 students and 
their teachers. The evaluation design was based on the Key Features Model of 
J. S. Renzulli (1975) using a quasi- experimental pretest-posttest control 
group design. Data sources included student and teacher questionnaires and 
structured interviews with students and teachers. The conflict resolution 
program, which featured accepting and respecting difference and skills for 
self-control and communication, was delivered over 10 weeks. Observations 
supported the positive effects of the program on school climate. Students 
reported increased self-confidence and an increase in the use of skills 
specific to conflict resolution. Teachers reported a decrease in interference 
with teaching, but the value perceived by teachers for the program did not 
change with the intervention. Implications for future evaluations of violence 
prevention programs are discussed. (Contains 5 tables and 39 references.) 
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Abstract 



Schools and communities throughout Canada struggle to address an increasing concern 
about violence among children and youth . The struggle involves selecting strategies that will 
be effective in making schools conducive to learning and in reducing violence. Equipping 
decision makers with information about the options available to them and providing them with 
empirical evidence of program effectiveness are valuable contributions that educational 
researchers and evaluators can make toward reducing violence in our society. 

The specific purpose of this study was to (a) determine the effects of a conflict 
resolution program on the school climate, the students’ self image, the use of conflict resolution 
skills, and the social validity of the program, (b) identify the benefits and limitations of conflict 
resolution in comparison with other violence prevention programs, and (c ) explore methods of 
evaluation for violence prevention programs. The focus of this study was the evaluation of a 
conflict resolution pilot project organized by four community service organizations and funded 
by provincial and federal government agencies. The context of the study was grade five 
classrooms of three French and four English elementary schools in two urban areas of a large 
Canadian city. The participants of the study included students (140), home-room teachers, and 
other school teachers with responsibility for the participating classrooms. 

The evaluation design was based on the Key Features Model of Renzulli (Renzulli, 
1975). Data collection procedures used a quasi-experimental pre-post test control group 
design. The collection of data with multiple instruments from several sources permitted 
quantitative and qualitative data analyses. Between methods triangulation enabled the use of 
strong points from each type of data, cross checking of data, and collection of information that 
was only available through particular techniques. 
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Finding Evidence To Support Violence Prevention Programs 

Schools and communities throughout Canada struggle to address an increasing concern 
about violence among children and youth (Day, Golench, MacDougall, & Beals-Gonzalez, 1995; 
Gabor, 1995). The struggle involves selecting strategies that will be effective in making schools 
conducive to learning and in reducing violence. Responsible organizations are ill equipped to 
make these selections because of a lack of understanding about the array of programs and 
confusing statistics about violence and the effectiveness of prevention initiatives. Equipping 
decision makers with information about the options available to them and providing them with 
empirical evidence of program effectiveness are valuable contributions that educational 
researchers and evaluators can make toward reducing violence in our society. 

There are no clear trends about either the extent of the violence in schools or the 
effectiveness of current intervention strategies used in Canada and in the United States. Some 
studies report an increase in school violence (Federation, 1994; Roher, 1993; Ryan, Matthews, & 
Banner, 1993) while others report a decline (Cusson, 1990; Fitspatrick, 1994; Rogers, 1993; 

West, 1993). Educational evaluators at this time report uncertainty about the effectiveness of 
programs designed to deal with the violence (Johnson & Johnson, 1995; Lam, 1989; Webster, 
1993; Wilson-Brewer, Cohen, O'Donnell, & Goodman, 1991). Some of this uncertainty arises 
because of the paucity of evaluations studies and the methodological weakness in the designs 
(Flannery, 1996; Levesque, 1996). In a survey of 210 Canadian school boards, Day (1995) found 
systematic program evaluation was the exception. Only 4.3% submitted a completed evaluation of 
a program currently in use. 

Apart from the need to have valid and reliable data on the effectiveness of programs, 
policy makers, funding councils, health and education services, and the general public need 
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assurances of the effectiveness of programs that are in place. Evaluation of conflict-resolution and 
violence-prevention programs can provide data for the service providers and consumers by 
determining whether goals are being accomplished and efforts are worthwhile 

Purpose 

The specific purpose of this study was to (a) determine the effects of a conflict resolution 
program on the school climate, the students’ self image, the use of conflict resolution skills, and 
the social validity of the program, (b) identify the benefits and limitations of conflict resolution in 
comparison with other violence prevention programs, and (c ) explore methods of evaluation for 
violence prevention programs. 



Theoretical Framework 

Two sets of theories formed the basis of this study. The first set included the literature on 
violence (Baron & Richardson, 1994; Eron, 1994; Goldstein, 1994; Heusmann, 1994), conflict 
resolution (Bodine, Crawford, & Schrumpf, 1994; Johnson & Johnson, 1989; Johnson & Johnson, 
1995; Katz & Lawyer, 1993), and violence prevention (Busby, 1996; Day et al., 1995; Goldstein 
& Glick, 1994; Henggeler, Melton, Smith, Schoenwald, & Hanley, 1993). Understanding the 
causes of violence is crucial to determining appropriate solutions (Day et al., 1995; Landen, 

1992). The solution goals provide a basis for subsequent evaluation. The second set of theories 
was drawn from the evaluation literature (Creswell, 1994; King, Morris, & Fitz-Gibbon, 1987; 
Patton, 1990; Popham, 1993) and more specifically the evaluation of violence prevention and 
conflict resolution programs (Day et al., 1995; Gabor, 1995; Johnson & Johnson, 1995; Lam, 
1989; Webster, 1993; Wilson-Brewer et al., 1991). 
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Method 

The methods used in this study were organized around (a) determining the effects of the 
conflict resolution program, and (b) comparing the program with other violence prevention 
programs. The methods and results will be discussed in that order. 

Context 

The focus of this study was the evaluation of a conflict resolution pilot project organized 
by four community service organizations and funded by provincial and federal government 
agencies. The project was in the second of a three year funding cycle and the study sought to 
determine the value of the program and to assist decision makers in the final year as to whether a 
request for future funding was justified. The project managers were not interested in finding 
positive evidence, rather wanted to truly know if the program was working and what direction it 
should take. 

The context of the study was grade five classrooms of three French and four English 
elementary schools in two urban areas of a large Canadian city. The participants of the study 
included students, home-room teachers, and other school teachers with responsibility for the 
participating classrooms. In all 140 students were involved. Eighty students in four classrooms 
comprised the program group and sixty- students in three classrooms comprised the control 
group. The control schools were matched and selected for similar socioeconomic, multicultural, 
and academic characteristics. 

Two groups of teachers who participated in the study were selected to complete two 
questionnaires. The first group included the teachers with primary teaching responsibility for 
classrooms participating in the study. Seven home-room teachers with experience ranging from 



Violence Prevention 5 



12 to 28 years, with the mean being 17 years, participated. Thirty ‘other 5 teachers participated in 
the second group. The ‘other’ teachers taught classes such as physical education, computer 
sciences, music, or arts. Home-room teachers completed both questionnaires. Principals of all 
schools participated in the second ‘other 5 teacher group. 

Design 

The evaluation design was based on the Key Features Model of Renzulli (Renzulli, 1975). 
Four key features emerged from the synthesis of data gathered in an evaluability study (Rossi & 
Freeman, 1993; Smith, 1989; Wholey, 1977) conducted as the front end analysis: school climate, 
self image of the student, use of skills, and the social validity of the program. These features 
served as the basis upon which change was measured. 

The fourth key feature, social validity, merits further clarification in order to understand 
its inclusion in the study. Social validity can be defined as consumer satisfaction with a social 
program (Khattri, 1991). The rationale for evaluating the social validity of a program is linked 
with the need to examine reasons as to why participants use or do not use program content. 

Social validity is concerned with three aspects of interventions: (a) are the goals important to the 
participants?, (b) are the consumers satisfied with the procedures of the program?, and ( c) are the 
consumers satisfied with the outcomes, including unintended outcomes? Social validity is based 
on the belief that participants who value and like a program are more likely to use the program 
knowledge and skills which the program promotes. 

Data collection procedures used a quasi-experimental pre-post test control group design. 
In order to control for threats to the validity for such effects as maturation, three classrooms were 
included as the control group in the design (Campbell & Stanley, 1963). In selecting the control 
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group, it was necessary to establish comparability for age, sex, race, socioeconomic, academic 
level, and school discipline records. As recommended by Lam (1989), the control group was 
selected in other schools to avoid interference. 

Data Sources 

The sources of data included (a) a questionnaire completed by students pre- and post- 
program, (b) a questionnaire completed by the home room teacher on each student pre- and post- 
program, ( c) a questionnaire completed pre- and post-program by all teachers with any teaching 
responsibility for the participating classrooms, (d) structured interviews with teachers pre- and 
post- program, and (e) structured observations of students pre- and post- program. 

The measurement instruments employed in the evaluation addressed recommendations 
made in the literature for multiple measures of effects and also addressed the deficits found in 
other studies (Cohen, 1989; Lam, 1989; Wilson-Brewer et al., 1991). The instruments were 
designed to systematically gather data. For example, interviews responses and observations 
collected in this fashion became data not anecdotes. 

All instruments were examined for content validity by seven professional colleagues, the 
content experts, a group of students of similar age and background to the experimental group, and 
finally a professional for final editing. Pilot interviews and observations were conducted with 
students and teachers using respective questionnaires. 

The collection of data with multiple instruments from several sources permitted 
quantitative and qualitative data analyses. Between methods triangulation (Smith & Kleine, 1986) 
enabled the use of strong points from each type of data, cross checking of data, and collection of 
information that was only available through particular techniques. Statistical significance found for 
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features was supported with evidence from interviews and structured observations increased in 
validity. 

Coding Procedures 

The four key features which served as the basis for the design and subsequent 
measurements also were the basis of the coding procedures used in analysis. For example, 
questionnaires completed by students and teachers sought responses regarding the key feature 
‘use of skills’. Students were asked to identify the “ways they dealt with arguments with peers”. 
Teachers were asked to describe the ways each student typically dealt with conflict. ‘Other’ 
teachers were asked to describe methods students used in conflicts with peers. Observations were 
structured to collect frequencies of the various ways students used in conflicts with peers in the 
classrooms, and during recreation periods. Interviews were structured to solicited the same 
information. Data from all three questionnaires, the interviews, and the observations were coded 
in three categories: negative, positive, or specific to conflict resolution skills. Inter-rater reliability 
checks on coding were conducted. Frequency counts within the coded categories were then used 
in matched t-tests. 

As a second example, the key feature of school climate was examined by asking students 
whether conflict was a problem for them in school and asking them to describe the “climate”. 
Teachers were asked the same question. Interviews also were structured to obtain teachers 
descriptions of the class and school climate. Observations required that the climate be described in 
low inference behaviors for each observation. Data was coded into two categories, (a) negative, 
when terms such as tense, stressed, aggressive, volatile, or disruptive were used in descriptions, 
and (b) positive when terms such as cooperative accommodating and positive were identified. 
Frequency counts were analyzed using t-tests. All classrooms, in experimental and control groups 
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were compared pre and post program. Further triangulation was accomplished for each key 
feature by using several questions. 

Program Comparison 

The methods used to conduct the comparison of the conflict resolution program to other 
violence prevention initiatives involved a literature review and document analysis. Alternative 
programs were researched in order to reveal the various perspectives, benefits, and limitations 
rather than identify a ‘right way’. The goals, content, and instructional strategies were identified 
and categorized to highlight the similarities and the differences. Findings arranged in dimensions 
and continuums would serve to educate the decision-makers about characteristics of the 
programs. On the basis of this comparison decision-makers would be better equipped with 
information to assist in determining the future of the project under evaluation. 

Characteristics of the Conflict Resolution Program 

The project that was being evaluated is delivered over ten consecutive weeks with a 
follow-up class on the twelfth week (i.e. following a two week gap). The final class period is used 
for evaluation and for facilitating transfer of skills. The content topics included in the program are: 

(a) the nature of conflict 

(b) accepting and respecting difference; awareness of ourselves and others 

(c) finding ourselves in conflict: conflict styles 

(d) skills for effective communication 

(e) taking control of ourselves: understanding feelings in conflict 

(f) Resolving conflict 
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The project employs one animator who delivers the program. The animator is responsible 
to the prevention committee and must work cooperatively with the teachers and the school. The 
animator must possess the skills to organize schedules to fit within four schools, to work in a 
multicultural setting, and in two languages. Teachers remain in the classrooms as resource persons 
and assist in managing the class. Teachers retain accountability for the class and students. 

Results and Discussion 

The results of the study are presented in relationship to the key features used as measures 
of change. 

School Climate: Conflict in the Schools 

Students and teachers agreed that the occurrence of fights, arguments, and conflict was a 
problem. This agreement was consistent for schools in the program and control groups and in pre- 
and post-tests. No significance was found in statistical analysis of responses to questions about 
perceived problems. In pre-tests, 93% of the program group and 91% of the control group 
identified that they had to deal with fights and arguments (see Table 1). Post-tests revealed this 
concern remained about the same. While the program group response dropped to 85%, this 
decrease was not found to be a significant reduction and indeed continued to indicate a very high 
level of concern. Teachers also agreed that conflict was a problem for students in their schools ( 
88% pre and 89% post). 

Students in all schools agreed with the statement that youth violence was a problem in 
their schools or neighborhoods (Table 1). This belief about the extent of the problem increased 
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Table 1. 

Student perceptions of conflict, (N=140) 



Variable 



Pre-test Post-test 

Control Program Control Program 



Must deal with fights, conflict 91% 93% 92% 85% 

Youth violence in school, neighborhood 71% 66% 75% 72% 



significantly for students in post-program tests of both the control and program groups, t_(138) = 
2.295, p < .023. This suggests that the concern about youth violence could be a daily and 
escalating concern for these students. 

The reported extent of problem arising from conflict was supported by teacher ratings of 
the level of conflict and aggression (see Table 2), by observations of students behavior in and out 
of the classroom, and by interview responses of teachers and students. Fifty-seven percent of 
teachers in pre-program questionnaires rated the level of conflict and aggression as high on a 
three point scale. Only 7% rated the class conflict as low. Fifty percent of teachers indicated that 
conflict in the class interfered with teaching time everyday with 71% occurring in the range of 1-4 
times during a typical class. 

Prior to the program, the climate of this class was described by teachers in terms such as 
tense, stressed, aggressive, volatile, or disruptive (75%) (see Table 3). A third of the classrooms 
(29%) felt this climate was getting worse or unchanged (42%). Following the program, terms 
such as cooperative, accommodating, or positive increased in post-tests of the program schools 
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(25%-42%). Teachers involved in the program group post-tests more frequently described the 
classroom climate as improving (29%-61%). While program participants responded more 
optimistically, control group participants remained unchanged and continued to describe their 
classrooms in negative terms. 

Table 2 

Teacher perception of conflict, (N=30) 



Variable 




Pre 

Control 


Program 


Post 

Control 


Program 


Conflict is a problem for students 


85% 


88% 


88% 


89% 


Level of conflict for this class 


High 


58% 


57% 


66% 


22% 




Average 33% 


36% 


33% 


44% 




Low 


9% 


7% 


0 


34% 


Interferes with teaching 


Every day 


58% 


50% 


50% 


17% 




Every week 


33% 


29% 


42% 


33% 




Every month 


9% 


14% 


8% 


39% 




Never 


0 


7% 


0 


11% 


Frequency of conflict 


5+/class 


8% 


7% 


8% 


6% 




1-4/class 


58% 


64% 


66% 


39% 




weekly 


25% 


21% 


25% 


33% 




monthly 


8% 


8% 


0 


22% 
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Observations of school classrooms and interviews with teachers following the program 
supported the questionnaire results that school climate had shown improvement. Classrooms were 
described as more respectful of others, with more listening occurring, and students taking greater 
responsibility for their behavior. One teacher spoke with appreciation of the occasions when 
students solved their own problems and the class was “even peaceful”. 

Table 3 



Descriptions of Classroom Climate 



Descriptions 


Pre 

Control Program 


Post 

Control Program 


Negative terms 


75% 


74% 


75% 


58% 


Positive terms 


25% 


26% 


25% 


42% 


Improving 


25% 


29% 


33% 


61% 


Getting worse 


33% 


29% 


33% 


0 


Consistent 


33% 


42% 


33% 


33% 


Impossible 


9% 


0 


0 


6% 



Student Self-image 

Students chose responses in the questionnaires demonstrated behaviors in the classroom, 
and made statements in the interviews which indicated lack of confidence, low self esteem, and 
anger. In pre-program data collection students chose aggressive reactions to conflicts and difficult 
situations with friends and family. Hitting, fighting, making faces, shouting, and cursing were 
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common responses. Thirty-three percent of student responses were coded as negative. Many 
students reported feeling victimized, threatened, trapped, and ignored. Their reactions to these 
situations were to cry, react by lying, fight back, or isolate themselves respectively. Ten percent of 
students revealed that they did not tell anyone including their parents how they felt or what they 
thought. Interviews suggested students believed no one cared, understood, or would listen. 

Taking “an Advil” was an acceptable coping mechanism when difficulties were encountered. 

In pre-program questionnaires, 24% students rated their ability to deal with conflict as 
‘bad’ and 18% rated it as “OK” or “good”. In post-test questionnaires, student’s self ratings of 
their confidence improved. Fifty-eight percent of students rated their ability as OK, good, or veiy 
well. Only 3% of students rated their ability as “bad”. This increase in confidence was reflected in 
prosocial responses reported by students in post-program data. Students reported that they were 
communicating their thoughts and feelings to friends and family in conflict situations. 

Observations of students in the classroom and during recreation periods supported what was 
being reported attempts. Teachers also reported an increase in prosocial behavior and confidence. 

Use of Conflict Resolution Skills 

In all of the experimental classrooms, students showed a significant increase in the use of 
skills specific to conflict resolution. Students responses showed significant increase in use of 
conflict resolution and prosocial skills with peers, t (79) = 4.501, p < .001 and with parents, 
t (79) — 5.514, p < .001. Teachers also reported a significant increase from their perspective, t 
(15) = 3.529, p < .003. Interviews with teachers supported this finding. “I hear the words as they 
[students] try to solve their own disagreements. The most important thing is that they are aware, 

. . . aware of their options now”. 
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Social Validity 

According to the students, perceived usefulness and appreciation of the conflict resolution 
skill training increased from 59% in pre-tests to 89% in post-tests. Students enjoyment of the 
procedures and instruction increased from 66% to 85%, t (79) = 3.938, p < .001. In post program 
interviews students reported wanting to continue in a similar program and stated that they had 
recognized changes in themselves from the beginning of the program. 

The same appreciation was not found with the teachers. Despite consistent teacher reports 
of an improved class climate and a decrease in interference with teaching, the perceived value of 
the program by teachers was unchanged from pre- to post-program conditions (see table 4). 



Table 4. 

Teacher Reports of Social Validity of the Conflict Resolution Program. 



Response 


Pre 


Frequency 

Post 


Useful 


50% 


56% 


Not useful 


6% 


6% 


Undecided 


44% 


39% 



This response to the value of the program may be the result of several factors. First, only 
four of the eighteen teachers were in the class while the program was delivered. Second, during 
the interviews, many teachers said they thought the length of time was inadequate to “make a 
difference”. Third, immersed in the setting each day, teachers may be unable to see a response to a 
twelve week program. Fourth, several teachers reported seeing “things (programs, services) come 
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and go” and “nothing helps us”, many just adding to the work. Seventy-five percent of teachers 
indicated that the conflict among students was a problem for them as teachers. Fights, disruptive 
behavior, and conflicts all interfere with their “job”. In the interviews, they reported feeling 
frustrated, tired, and sometimes “angry (them)selves”. 

Program Comparison 

The evaluation served to meet the committee’s needs in six ways. First, they gained better 
insight as to the effectiveness of the program. The evidence of these effects supported their efforts 
to initiate conflict resolution programs for future years and classrooms. The evidence also 
provided data to convince others of the programs merit. Second, the comparison of the program 
with other programs educated the committee and other allied parties about the types of violence 
programs being offered elsewhere, the extent of youth and school violence in schools in their 
community and across the country, and their part in a multisystem approach to violence 
prevention. Third, the evaluation outlined the alternatives for future program expansion and 
improvement. The identification of areas for possible integration with other agencies, services, 
and systems were seen as valuable. The committee spent considerable effort determining its 
position on the various dimension that could be adopted (see Table 5). Fourth, resources were 
identified to assist the animators, the committee, and the community develop prevention 
measures. Connections and networks were developed with other agencies, and federal and 
international educational and research organizations. For example research done by the Solicitor 
Generals Office in Ottawa provided needed information of other studies and Canadian statistics 
[Day, 1995 #87][Gabor, 1995 #170] and the National Association of Mediation in Education 
(N.A.M.E.) in Washington offers many types of educational resources. This lessens the isolation 
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Table 5. 

Dimensions of Programs & Policies for Violence Prevention Programs 



Reactive 



4- 



Programs & Policies 



Target 

Few aggressive 4- 

Older adolescents 4- 



Single focused 



4* 



Outcome 



Decrease in 4- 

aggressive 

behavior 



— -» 


Proactive 




All & staff 




Younger 




Wide range 




Increase in 




prosocial 

behavior 



felt by organizations as they deal with complex social issues such as violence. Fifth, marketing 
tools were developed for the committee. A brochure, an evaluation report in three forms, and 
audiovisual material assisted the committee to speak to others about the project. Finally, the 
evaluation assisted in securing funding from one federal source. While funding is often a major 
concern for such programs, the committee now reports feeling better equipped to address this 
task. 

Conclusions 

The major implications as a result of this study for future evaluations of violence 
prevention programs include; 

• Equipping our communities with the knowledge, resources, and confidence to tackle 
important social issues such as violence prevention is possible. Researchers and 
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evaluators should consider a broader role for themselves, one which can support 
decision-makers beyond conducting studies. 

• It is challenging to establish the effectiveness of programs that are preventative in 
nature. How does one measure a reduction of something that did not occur? 
Nonetheless, it can be done. Creative and skillful evaluations can provide valid and 
reliable data on program effectiveness. 

• Teachers are essential to the success of violence prevention initiatives and they need 
to be supported in their role in educating our youth. Integrating teachers in violence 
prevention programs is important in the transfer and consolidation of the skills and 
knowledge. 

• Complex activities are needed to address the complexity and multifaceted nature of 
violence in our society. This can be attained in integrating programs. Integration must 
occur at many points. First, conflict resolution programs need to be integrated into the 
school academic and social life. Add-on programs are limited by the perception of 
marginal status compared to the core programs. Second, principles and skills 
underlying conflict resolution could be integrated in the curriculum. Current research 
is paving the way for this to happen (Stevahn, Johnson, Johnson, & Real, 1996). 
Finally, conflict resolution must be integrated and supported along with other violence 
prevention initiatives throughout the school and greater community and country. 
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